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ABSTRACT 



To incorporate ideas from the sociology of science 
into the policy-making activities of the American Educational 
Research Association, a colloquium was held consisting of AERA 
Dlanning committee members, executive officers, council members, and 
scholars from the sociology of science. This colloquium was 
specifically designed to help the Planning committee members make 
recommendations to the council. Among the major Ideas coming from 
these meetings were the following: Two resulted in favorable 
action— new AERA publications, and improvinq relations with the 
Federal Government: two are pending additional inf ormation--improving 
the research climate in schools of education, and initiating research 
in educational development: and one appears to have been permanently 
tabled-association reorganization. Committee members indicated that 
the colloquium satisfactorily accomplished its objective of providing 
ideas to the committee. (RA) 



ON 

fN^ 

- 3 - 

Q . 

Q 

UJ 



o 

w 

H 

CO 

o 

o 

«c^ 

w 




AN EVALUATION OF A STRATEGY FOR BRINGING SOCIAL SCIENCE 
TO BEAR ON ORGANIZATIONAL DECISION-MAKING* 



Caroline H. Persell 
Bureau of Applied Social Research 
Columbia University 
New York, N. Y. 10025 

April 1970 



U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE 
OFFICE OF EDUCATION 



THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECES5ARILY REPRESENT OFFICIAL OFFICE OF EDUCATION 
POSITION OR POLICY. 



A preliminary version of this report was given at the 1970 A'ERA meeting 
as the paper entitled, "Ideas from Social Science Applied to Organizational 
Policy Making : A Case Study of AERA." 

This report was supported by a grant from the U. S. Office of Education, 
through the American Educational Research Association (AERA) . 

With the advice of Sam D. Sieber, Columbia University, New York, N. Y. 




ZfxW 0Q3 



AN EVALUATION OF A STRATEGY FOR BRINGING SOCIAL SCIENCE 
TO BEAR ON ORGANIZATIONAL DECISION-MAKING 



Caroline H. Persell 
Bureau of Applied Social Research 
Columbia University 
New York, N. Y, 10025 

April 1970 



I. PURPOSE AND METHODS 

The purpose of this report is to evaluate a method of bringing 
ideas from the sociology of science to bear on American Educational 
Research Association (AERA) policy-making. The method under study is a 
Colloquium of scholars in the sociology of science. The Colloquium was the 
final phase of a strategy for bringing social science ideas to bear on the 
problems of research on education, particularly those problems which might 
be within the control of a major professional association in the field, ^ 

A 

At the two day November 1968 Colloquium, scholars^’ presented papers 
and discussed ideas with AERA leaders — AERA Council members. Long Range 



For the results of earlier phases of the strategy, see Ronald G. 
Corwin and Maynard Seider, "Patterns of Educational Research:. Reflections 
on Some General Issues," one of the papers prepared for the Colloquium# 

2 

The following papers were presented: Corwin and Seider, op. cit 0 ; 

Garvey, 'William D„, Carnot Nelson, and Nan Lin, "A Preliminary Description 
of Scientific Information Exchange in Educational Research"; Hagstrom, 
Warren 0,, "Educational Researchers, Social Scientists, and School Profes- 
sionals";’ Lingwood, David A., "Interpersonal Communication, Scientific Pro- 
ductivity, and Invisible Colleges: Studies of Two Behavioral Science 

Research Areas"; Paisley, William, "The Role of Invisible Colleges"; and 
Storer, Norman W. , "The Organization and Differentiation of the Scientific 
Community: Basic. Disciplines, Applied Research, and Conjunctive Domains." 
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Planning Committee (LRPC) members, and executive officers,,^ 

The Colloquium mil be evaluated on two dimensions: (l) the ap- 

plicability of social science ideas for the situation AERA confronts, and 
(2) the impact of the Colloquium on the governing bodies of AERA, as medi- 
ated by the Long Range Planning Committee (LRPC), 

The planned sequence of decision-making for AERA was as follows: 
LRPC would meet several times after the Colloquium and draft a report with 
policy recommendations for the organization. Then the AERA Council would 
meet to consider the LRPC recommendations. Finally, appropriate actions 
might result from their decisions. 

We used the following research methods in this evaluation. First, 
the researcher observed AERA Executive Committees and Council meetings, 
and a meeting of the AERA Federal Advisors, Where appropriate, these meet- 
ings were tape recorded. Second, the researcher did a content analysis 
of papers presented at the Colloquium, the “Recapitulation of the Col- 
loquium," and of the Long Range Planning Committee (LRPC) Interim Report, 
Third, after hypothesizing some relationships between the Colloquium and 



Besides the paper-givers, the following people attended the Col- 
loquium: Simon Marcson, Sociologist of Science, Rutgers University; Lee 

Burchinal, Bureau of Research, Office of Education; Arthur Lumsdaine, Psy- 
chologist, University of Washington, observer from the American Psychologi- 
cal Association; John Mayor, from the American Association for the 
Advancement of Science; Ray Norris, Office of Education; Theodore Parsons, 
University of California, Berkeley, observer from the American Anthropologi- 
cal Society; Garry Walz, University of Michigan, ERIC Clearinghouse on 
Personnel Services; LRPC members: Charles Bidwell, University of Chicago; 

John Goodlad, UCLA; Nate L , Gage, Stanford University; Leslie McLean, 

Ontario Institute for Studies in Education; Richard Schutz, Southwest 
Regional Laboratory for Research and Development; B, Othanel Smith, Univer- 
sity of Illinois; AERA Council: Frank Besag, State University of New York 

at Buffalo; David R. Krathwohl, Syracuse University; Robert M.W. Travers, 
Western Michigan University; AERA Staff Members : Richard Dershimer, Execu- 

tive Officer; Gary Hanna, Assistant for Professional Affairs. 
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the LRPC Report on the basis of the content analysis, the researcher inter - 
viewed members of the LRPC, to see if they perceived the ideas as related 
to the Colloquium,^ - 

This evaluation attempts to trace the progress of certain ideas 
from the Colloquium through the decision-making structure in AERA, The 
process is analogous, although surely not identical with, tracing radio- 
active isotopes through the body, to see what happens to them at different 
places o 



II, MAJOR THEMES FROM THE COLLOQUIUM AND THE 
STRUCTURE OF DECISION-MAKING 

Several themes emerged from the Colloquium which seemed relevant to 
AERA policy makers. Very briefly these include: 

A, The Need for a Stronger Community of Scholars , 

1. to provide adequate social rewards for people doing 
research on education, 

2. to improve the communications mechanisms within the 
field of research on education, specifically at the 
annual meeting and through Association publications, 

3. to support "basic" research in the face of strong 
pressures for research with immediate applicability. 

3. The Importance of Development People in educational 

research. Efforts were made to define "development," and 

^All six members of the LRPC who attended meetings were interviewed, 
except the Chairman, He was on sabbatical and out of the country or other- 
wise unavailable during the time of the interviews. 

'’For more details see the "Recapitulation of the Colloquium" 
or the actual papers presented at the Colloquium, which AERA 
hopes to have published. 
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there was strong sentiment that this important new breed 
must be recognized and included in the AERA. 

C, The Importance of the Climate for Research in Schools of 
Education for fostering or impeding research on education. 

D, The Importance of increasing the Understanding and Support 
of the Larger Community for research on education, particu- 
larly that of the federal government. 

E, AERA was urged to write a scenario for itself — conceptu- 
alizing its long-term objectives and developing strategies 
for achieving them. 

Clearly there were ideas in the Colloquium which are relevant to 
the situation AERA faces. In terms of the first dimension for evaluating 
the Colloquium, then, we can conclude that the Colloquium did yield appli- 
cable ideas for the problems ‘faced by AERA. 

The structure of decision-making within the AERA can best be 
described schematically (Figure l). The Colloquium was designed to pre- 
cede and to assist the LRPC in its deliberations and recommendations. The 
LRPC in turn ms to report to the AERA Council, which has the constituted 
authority to act on their recommendations. We will consider the transmis- 
sion of ideas at each step in this decision-making process, and then make 
some observations about the overall process. 
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FIGURE 1 

THE STRUCTURE OF DECISION-MAKING WITHIN THE AERA 
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III. "IMPACT 1 ' OF THE COLLOQUIUM ON THE LRFC 

We will examine what happened to each of the main themes which 
emerged from the Colloquium when they were considered by the Long Range 
Planning Committee (LRPC). 

A. Community of Scholars 

1. The LRPC recommended that a special membership category, that 
of Fellow, be established. By this they meant a membership category 
signifying some degree of excellence in furthering the work of the field, 
as is practiced by other scholarly associations, such as the American 
Psychological Association.^ The LRPC member who, with someone else, 

^See pp. 7-13 of the LRPC Interim Report for more 
details on this recommendation. 
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formulated this idea said, "I hesitate to say that it was a result of the 
Colloquium, although the Colloquium did discuss ideas about the reward 
structure of the field and socialization patterns. The Colloquium helped 
generate more ideas and clarified other ideas," 

2. Regarding publications. One LRPC member thought "the publica- 
tions recommendations are the clearest example of the influence of the 

Colloquium." The LRFC report had a number of specific recommendations 
7 

about publications. One of these was the recommendation that a critical 
Review of Educational Research be started. Several LRPC members commented 
that they weren't sure the Colloquium could be credited with this idea, 
since certain committee members had favored it for some time. The data and 
opinions from the Colloquium certainly supported this idea, however, 

3, Regarding meetings. The IRPC recommended that AERA experiment 
with smaller, more frequent meetings by divisions or geographical regions. 
Also, they urged continued analysis of all meetings. One LRPC member said 
the latter idea "could be pinned doxm as resulting from the Colloquium," 
since the Colloquium demonstrated the value of data about annual meetings 
and, more generally, the communication structure of the organization. 

U. Regarding the social organization of the field. The Colloquium 
raised questions about the nature of AERA's membership. The LRPC spent a 

Q 

lot of time discussing the divisional structure 0 of AERA, and as one member 

^See "Directions for the American Educational Research Association; 
Interim Report of the Long-Range Planning Committee," pp, 3-6, 

R 

°As an Association, AERA is organized into seven divisions. These 
are: Administration, Curriculum and Objectives, Learning and Instruction, 

Measurement and Research Methodology, Student Development and Personnel 
Services, History and Historiography, and Social Context of Education, Each 
of these divisions is represented on the AERA Council by an elected Vice 
President, 
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said, "considering the consequences of alternative ways of organizing AERA, 
e.g., how they might involve more people from the disciplines." He felt 
that ideas on the membership and internal structure of AERA were "crystal- 
lized through the Colloquium." 

B, Relations with the Larger Community 

The IRPC made five suggestions on this subject: 

1. That the AERA work collaboratively with other groups in an 
effort to strengthen public and Congressional support for educational re- 
search. 

2. That the AERA provide for thorough, quick review of educational 
projects financed by the Federal government. 

3o That the AERA form a continuing series of ad hoc committees 
addressing themselves to educational policy issues which could be illumi- 
nated by research processes and findings. 

U. That the AERA serve as a kind of clearing house for the identi- 
fication of individuals willing and able to provide expert testimony before 
Congress on various educational problems and issues. 

!?• That the AERA create a standing committee both to examine 
federal policy for education and to provide the membership with relevant 
data on federal and other commitments to educational research. 

As one committee member noted, "people were aware of the importance 
[of relations with the larger community, especially the federal government] 
before, but the Colloquium may have made more converts to the point of vievr 
of AERA doing more in the area." This is an example of an attitudinal rather 
than a cognitive result of the Colloquium, 
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C, Climate for Research in Schools of Education 

The LRPC recommended that the AERA Council appoint a task force to 
examine the place of and the conditions for educational research in univer- 
sities offering graduate work in education. As one LRPC member observed, 
"Hagstrom 1 s paper highlighted the problem of the climate of schools of edu- 
cation. It's not a new problem. People were aware of it before." 

D. Educational Development 

The LRPC Report barely mentions this issue, which made me wonder if 

*i * 

they had considered it. Several members mentioned that they had spent a 
good deal of time on the problem. One said that he and another member had 
tried to gather data on developers — the type of people they were, the work 
they do, and whether they were in AERA or not. "We had trouble agreeing 
who they were, what they do. The time we spent couldn't solve the problem. 

It would still be a good study to do." 

In brief, there is remarkable overlap between some of the ideas in 
the Colloquium and certain LRPC recommendations. 

IV. COUNCIL'S DECISIONS ON THE LRPC RECOMMENDATIONS 

We will now consider what happened to the themes from the Colloquium, 
as they were set forth in recommendations from the LRPC, when they were 
considered by the AERA Council. 

A. Community o f Scholars 

1. The membership category of? Fellows# The executive officer cited 
it as one of the "stickiest wickets" in the whole report. He said the 
Council might want to refer the idea to a committee. One member of Council 
said, "We discussed the Fellows in the June Council, I still feel the same 
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to y, I -want to know the LRPC 1 s rationale." This same member felt that the 
Fellows recommendation aimed AERA's priorities completely at scholars. 
Another Council member suggested that the Council might want to get a much 
broader reaction from the whole AERA membership on the idea of Fellows. 
Also, he felt they needed more information about the costs of such a plan. 
(He seemed to mean both financial and organizational costs.) He felt that 
the implications of such a suggestion needed to be probed. The Council was 
clearly not prepared to take action on this recommendation at the December 
meeting. They felt they should wait to see what the Ey-Laws Committee 
recommended about membership. Then they could decide what to do about the 
Fellows idea. 

2. Membership. After considerable discussion of this issue, the 
Council moved that an Ad Hoc Committee be appointed to suggest priorities 
in recruitment of new members and that the committee get advice from the 
divisions of AERA. One Council member asked if the Ad Hoc committee was 
empowered to act, and what would its functions be, to clarify issues or to 
come up with recommendations for Council? He suggested changing the word- 
ing of the motion to "determine" rather than "to suggest." In short, this 
issue was not decided by the Council, but referred to a new committee for 
their consideration. 

3. Publications. The Council devoted much discussion to this 
issue. The AERA staff remarked that it was clear that the Publications 
Committee did not feel they had a charge to develop a program. Recommen- 
dation 3 tos amended and passed by Council as follows:^ 

Q 

'See LRPC Interim Report, p. 3, for the original version of this 
recommendation. 

O 
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The Committee recommends that the Association develop -with delib- 
erate speed a publications program designed to (l) attract and dis- 
seminate the scholarly work of the entire educational research 
community; (2) provide efficient, informal communication among 
scholars of like interests within the organization and its several 
subdivisions; and ( 3 ) report and speak about educational research 
to a larger educational audience. These goals are listed in order 
of their priority. 

This recommendation was referred to the Publications Committee, 

Another Council member suggested that studies of publications read- 
ership were needed, and he observed that there was slippage with regard to 
where the responsibility lay for the Garvey and Paisley data. He seemed to 
be wondering whether the Publication Committee, the Council, c-r some other 
group should try to integrate the Garvey and Paisley data into policy- 
decisions 

In addition, the following IRPC recommendation was referred to the 

AERA Finance Committee by the Council: 

Recommendation J 4 .: The committee recommends that the Association 

Council set clear directions for a publications program based on 
the foregoing recommendations, give the Publications Committee all 
necessary authority for implementing this program, and take such 
steps as are needed to assure long-range planning for publications 
geared to the changing needs of. AERA membership. 

The Council seemed to feel that the Finance Committee should be involved 

because many decisions about publications depend on costs and availability 

of funds, 

lu Meetings, The LRPC recommended the following: 

The Annual Meeting increasingly should be directed to advancing the 
field of educational research through the presentation of exemplar 



•^Garvey, op , cit » , found that 20>6 AERA convention papers were sub- 
mitted to 67 different journals. From this he concluded that the communi- 
cations mechanisms in the field were not working efficiently, because there 
was no "core" of journals in the field of research on education which one 
could read to stay current with the field. 



reports of truly significant research,, critical analysis of re- 
search developments, symposia on frontier developments, and cross- 
discipline progress reports. 

The Executive Officer reported that these recommendations "were transmitted 

to the 1970 Program Committee and additional funds •were added to the budget 

n 

to implement some of the recommendations." The Program Committee was 
asked to report back with a cost estimate. In brief, the Council referred 
this issue to the Program Committee, and is awaiting action from them 
before doing anything itself. 

In a related vein, the Executive Officer mentioned studying future 
annual meetings, to find out, among other things, why people come. The 
major obstacle to doing this is obtaining funds for it. No one on the 
Council objected to this idea, 

$, Organizational structure. Many of the LRPC recommendations 
about organizational structure were referred by the Council to the By-Laws 

1 p 

Committee, specifically items i;-8 and 10-12 in Richard Dershimer ! s Memo. 
Furthermore, the Executive Officer suggested that recommendation# 9 in. 
his memo which deals with preparing an outline of formal responsibilities 
of divisional officers, could be referred to the AERA staff. This was 
done. Thus the Council allocated responsibility for membership decisions 
to other bodies 0 

B. Relations with the Larger Community 

Along the lines of the LRPC suggestions, the AERA Council made and 
passed the following motion: 

^See Richard Dershimer Memo, Appendix I, p. 3. 

■^See Appendix I, 
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The Council authorizes the executive officer to explore further the 
feasibility of establishing an affiliated organization under pro- 
vision 5dc6 (of the IRS code) and report back to the next Council. 
fThis type of organization is free to engage in lobbying activity 
with the government, since it does not have a tax exempt status. 

AERA as an organization is not free to lobby per se or it is sub- 
ject to losing its tax exempt status.] 

C, Climate for Research in Schools of Education 

The LRPC recommended that 

the Association Council appoint a task force to examine the place of 
and the conditions for educational research in universities offering 
graduate studies in the field of education. 

The Council had already authorized the President to appoint a Commission to 

study the consequences of how research funding programs were organized. He 

felt that the Commission should study the organizational structure of 

schools of education as well. In view of the existence of this Commission, 

Council took no further action on this LRPC recommendation. 

D . Educational Development 

The issue of the relation of development people to AERA was never 
considered specifically by Council, perhaps because it wa s not in the LRPC 
Interim Report. It arose throughout Council’s discussion of membership 
decisions, however, so it is evident they had it in mind, although no con- 
crete action was taken. 

B . General Reaction of Council to the 
LRPC Interim Report 

The Council tried to get a member of the LRPC to come to their meet- 
ing, and also to get a final version of the LRPC Report for the meeting. 
According to the Association President, "We didn't get either, through no 
lack of effort on our part, I feel some disappointment that the next steps 
were not taken." Another Council member remarked, "I feel a little reluctant 
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to plunge in on things when the committee hasn't made its final report.” 

The Executive Officer, however, expressed the fear that ”a six month delay 

will bring things to a screeching halt.” By this he seemed uo -gtean that 

■cp 

the issues which had been raised by the Colloquium and the LRPC were vital 
to the Association, and that if action were not taken on any of them, the 
earlier work might have been, in vain. This instance is but one example of 
the vital role played by the Executive Officer in the entire decision-making 
process. In terms of Figure 1, he was present at every stage of the 
process. More than that, he served as a very crucial linking person between 
the steps in the process. As illustrated in the example immediately 
above, his enthusiasm often made the difference between an idea being con- 
sidered and its being dropped, As in any large organization where inertia 
is often the rule rather than the exception, AERA might have done less if 
it had not been for the Executive Officer's energizing force. 

A final general result of the Council's consideration of the LRPC 
recommendations may have been the development of a new' style of operating 
the Association. After the Council had spent 8-10 hours of the two day 
meeting discussing the LRPC Interim Report, the Executive Officer observed 
that the emphasis in the Association had moved away somewhat from the LRPC 
type of approach for changing the Association, toward a tendency to work 
for changes and long-range planning through existing AERA committees. In 
this era of rapid social change, most organizations at least pay lip 
service to the idea of building mechanisms for adaptation and change into 
an organization. Whether AERA has succeeded in doing this remains to be 
seen over time. Also, whether or not this is a direct result of the Col- 
loquium is very difficult to determine, although it is a possibility* 
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V 0 PRESENT STATUS OF IDEAS FROM THE COLLOQUIUM 

We have seen what happened to some of the ideas which emerged from 
the Colloquium and the ERPC when they were considered by the Council. We 
will now consider the final stage in the decision-making sequence., namely, 
what actions, if any, have resulted from this process. We will consider 
these ideas in the same sequence as we did in earlier sections. 

A. Community of Scholars 

1. Publications, The first edition of a new critical Review of 
Educational Research is being published by AERA. Also, a critical Annual 
Review of Educational Research mil be published by AERA in 1972. The edi- 
tor will be announced by AERA momentarily. These actions are consistent 
with the Colloquium emphasis on the importance of an association raising 
the standards of research in its field, and with the ERPC recommendations 
in this vein. 

In addition, on Aprn 10, 1970, the Publication Committee recom- 
mended to the Council that AERA sponsor and develop a new social science 
journal, which would seek articles with original quantitative data, and 
also articles by historians and philosophers of education. In their June 
meeting the Council will consider this recommendation. This is a major 
decision which is consistent with the data presented by Garvey in the 
Colloquium, and with the ERPC recommendations on publications. 

2. Meetings. There is no real evidence that ideas from the Col- 
loquium or the ERPC are associated with new activities dealing with Asso- 
ciation meetings. Unlike the Publications Committee, there is almost com- 
plete turnover on the Program Committee, according to the Executive Officer. 
Therefore, he feels it is more difficult for that Committee to implement 
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the LRPC re commendation s « Also, he feels that the LRPC recommendations 
on this subject ''didn't really turn anybody on." For these reasons, it is 
not surprising that little new activity seems to have arisen from the Col- 
loquium and IRPC^s concern with the meetings of the Association. 

3. Organisational structure. One of the major concerns of the new 
President of AERA is the organizational structure of the Association, He 
feels that the whole issue needs some clearing up, and he hopes to work on 
this problem. It would be hard to say that this concern is the result of 
the Colloquium or the LRPC Report, The Executive Officer feels that the 
issue of organizational structure must be returned to from time to time. 

At any rate, this concern is consistent with the emphasis of the Colloquium 
and the IRPC. 

B, Federal Relations 

Consonant with the focus of the Colloquium and the suggestions of 
the LRPC, the Executive Officer has been more active since the Colloquium 
in informal conversations with key government policymakers. This also is 
difficult to attribute to either the Colloquium or the LRPC, although one 
member of the LRPC observed, "People were aware of the importance of the 
Federal Government before, but the Colloquium may have made more converts 
to the point of view that AERA should do more in the area," When asked 
about this, the Executive Officer indicated that he did feel more support 
for efforts in this direction now than he had in the past. This suggests 
an indirect effect of the Colloquium on the climate of support which exists 
for a particular type of action. 
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C 8 Climate of Research in Schools of Education 

The LRPC Interim Report recommended the formation of a task force 
to study the climate for research on education in schools and departments 
of education. The Committee to study this problem has not yet been offi- 
cially formed, although the idea for such a committee has certainly not been 
rejected. 

D. Educational Developmen t 

Although no official action was taken by the LRPC or the Council on 
the issue of educational development, there is some informal action in this 
area. 

The new President of AERA indicated last January that he would like 
the Council to spend one of its two days at the June meeting on the problems 
of educational development — what it is and what role AERA. should play in 
fostering and improving it. He is also concerned with the role the research 
person plays in the process of educational development. 

E. Summary of Outcomes 

Ideas which emerged from the Colloquium are at one of the following 
four stages: 

(1) In some instances real actions have resulted, as, for example,' 
in the case of new AERA publications which use a critical approach, 

(2) Sometimes no action has yet been taken, although it has been 
recommended, as illustrated by the Publications Committee^ recommendation 
for a new social science journal. These ideas can still result in actions. 

(3) Other ideas have not yet been acted upon nor have recommenda- 
tions for action been made. They have not, however, been rejected by the 
Association. At this point in time they are awaiting further consideration. 
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An example of this is the idea for a task force on the climate for research 
on education, 

(it) Finally, seme ideas were not acted upon, and future action 
seems unlikely. Recommendations for the annual meeting seem to be at this 
stage. 

In brief, there are certain ideas which emerged from the Colloquium, 
were incorporated by the LRPC, and accepted by the AERA Council. Some of 
these have already resulted in action by the Association, 

VI. OVERALL EVALUATION OF THE COLLOQUIUM 

We have seen some similar ideas in the Colloquium, the LRPC, and in 
present AERA activities. The question remains, were these ideas the result 
of the Colloquium or did they simply follow it coincidentally? The recur- 
rence of so many common themes suggests that the Colloquium had some influ- 
ence on the LRPC's ideas, although it is impossible to prove it conclusively. 
What we have done so far in this evaluation is to examine several major 
ideas which appeared in the Colloquium, to see if they reappeared in the 
later stages of the AERA decision-making process. If they did, we have 
briefly indicated what happened to these ideas at each stage. Thus we 
have established some sequence of associations. Association, of course, 
does not necessarily imply causation. It may suggest it, however, particu- 
larly if the associated ideas are so congruent that one has difficulty 
believing they occurred by chance. We can also examine two other sources 
of evidence to evaluate the Colloquium. These are comments of the LRPC 
members and the general impressions of this observer* if all three of 
these strands of evidence converge, the picture mil be more convincing 
than if we were relying on one strand alone. 
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When interviewed^ LRPG members said it was very hard for them to 
determine the causal effect of the Colloquium on their recommendations. 

In general terms, however, their assessments of the Colloquium’s worth 
were quite favorable. The following are all their relevant comments. Com- 
ments were deliberately not excluded, because it would be too easy to present 
a one-sided view that way, 

"The Colloquium was very helpful for getting information about 
communication processes. It was a very valuable operation,” 

"The Colloquium was very interesting. It set the stage for 
the IRPC. We worked much more effectively than if we hadn't had 
the Colloquium, The Colloquium stimulated and helped crystallize 
the ideas of the IRPC, It is hard to see exactly what specific 
ideas were informed by the Colloquium. This Colloquium was as 
effective as any could be, I'm skeptical about whether AERA is 
the place where you can intervene to change the nature or structure 
of a research field, [although] I've been glad to try to help do 
this. It won’t hurt, and it may help revivify the field." 

"The Coloquium provided useful background. It had an indirect 
rather than a direct impact on the work of the IRPC. The Storer, 
Hagstrom, Paisley, and Garvey papers were useful. The papers were 
short on what could be done. I thought the Colloquium was extremely 
worthwhile. It -was a good expenditure of money. I was impressed 
with the time and effort which went into the papers which were given 
at the Colloquium. I thought they were well done. The background, 
views, and data were all useful. The Colloquium may have converged 
some of our thinking, helped us find a framework for the salient 
issues. It saved us a lot of time." 

"I'm not very awrare of any explicit residue from the Colloquium. 

It was not relevant enough to the concerns of the IRPC. There was 
a kind of a forced marriage between • the Colloquium and the IRPC, 

The papers themselves were too voluminous. Maybe if we had had 
summaries, reminders, that would have been more helpful. It's my 
feeling that the Colloquium wasn't that fruitful. It may be my 
fault for not seeing the relation of the papers to the work of the 
IRPC. We didn’t need the technical data, the analysis of the 
communication researchers'." 

"Having the Colloquium was very much more helpful than not 
having it would have been, for the purposes of the IRPC. I would 
fund in an instant anything which added data. The papers pre- 
sented became data to us. Both the conceptual and the data papers 
were helpful." 
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While all but one of these comments presents a generally positive 
evaluation of the Colloquium 1 s worth, none of them present hard evidence 
for the indispensability of the Colloquium in the formulation of certain 
LRPC recommendations. Taken in conjunction with other imperfect evidence, 
however, this strand may contribute a part to the whole evaluation. 

A third strand of evidence is available in the general impressions 
of this observer. It seems to me that the Colloquium' s worth is not only 
in terms of specific ideas which were adopted by the Association, but, very 
importantly, in terms of the questions and problems it raised for AERA. 

Many of these general questions were not new, but the Colloquium reempha- 
sized their importance. Questions about the identity and purpose of AERA 
as an organization were raised throughout the Colloquium. These questions 
need to be answered before decisions can be made about recruiting new 
members, starting new publications, changing the structure of the organiza- 
tion, etc. This strand of evidence deals with a less cognitive domain of 
effects. It treats changes in people's attitudes as a result of the Col- 
loquium, As such, we cannot measure it objectively on a post hoc basis. 

It is the strong impression of this observer, however, that AERA policy- 
makers were faced with data and dilemmas which they had previously not had 
to confront so directly in their proceedings. The result was a sharpening 
of their awareness of the issues facing the Association. In my opinion, 
this outcome of the Colloquium was a very positive one. This indirect, 
attitudinal result is particularly evident in the attitude of the Executive 
Office, and was observed by both this researcher and by a member of the 
IRPC. He said, "the Colloquium had powerful effects on the Executive 
Office of AERA, mobilizing and energizing it." 




in brief, our evaluation of the Colloquium's worth depends on three 
strands of evidence. These are, first, the documented sequence of associ- 
ations among the stages in the decision-making processes, which may seem 
too similar to have occurred by chance. Second, the generally positive 
feelings of LRPC members about the worth of the Colloquium provide evidence 
in the same direction. Third, this observer's opinion of the positive 
value of the general questions and problems raised by the Colloquium, which 
intensified the AERA' s self-examination. Taken together, these three 
strands of evidence suggest the positive value of the Colloquium. 

In closing, it is interesting to speculate about why certain ideas 
seem to have been accepted and acted upon, while others were not, ¥e will 
now briefly consider this question, 

VII, WHY WERE CERTAIN IDEAS FROM THE COLLOQUIUM 
ACCEPTED AMD OTHERS NOT? 

A. There are several possible explanations for why ideas were 
accepted, 

1, Sometimes there was a perfect fit between the needs of the 
Association and the nature of the ideas, ^ For example, the Colloquium's 
emphasis on the importance of support from the outside community was con- 
sidered in a positive way by the Association, partly because it had been 
aware of the importance of this need before. The effect of the Colloquium 
may have been to intensify the importance. 

13 

This is similar to Ronald Havelock's "plus plus" model of the dis- 
semination of research in education. See Planning For Innovation, Ann Arbor 
Institute for Social Research, University of Michigan. 
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2, Some decision-makers of AERA were repeatedly exposed to ideas 
over time. They became familiar with certain themes through contact in 
the Colloquium, LRPC Report, and the Council meeting. For example, the idea 
of and importance of the reward structure in a social system was new to 
some people, but they were exposed to it repeatedly over time, and thus 
assimilated it into their way of thinking about the world, 

3« The Executive Officer of AERA became very familiar with the 
ideas presented in the Colloquium, by his own reading in the literature of 
the sociology of science, by attending the Colloquium, by reading the 
papers presented there, and by continually trying to bring these ideas to 
bear on problems facing AERA, If there was an interpreter of ideas from 
the Colloquium to the Council, he was it. At the Council meeting, he sum- 
marized some of the major themes of the Colloquium for the Council members. 

ko The Colloquium raised questions and documented needs which may 
have been only dimly felt before. They were accepted because they were 
based on previously unknown data. For example, Hagstrom's paper documented 
the influence of practitioners on the conduct of research in education, 
Garvey and Paisley presented hard data on communications patterns which 
could only have been surmised before, 

B, Other ideas were not accepted, or decisions on them were not 
made, for reasons which may or may not have to do with the intrinsic worth 
of the ideas themselves. There are several possible explanations for these 
results, 

1, Sometimes there was no advocate for an idea as it passed from 
one stage to another. For example, no one from the LRPC attended the Gouncil 
meeting, so there was no one there who could explain their rationale for 
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certain recommendations. This was particularly apparent in the reaction of 
the Council to the LRPC's recommendation for a membership category of Fellow. 

2. Certain ideas were very "sticky wickets o" Decisions, for 
example, about membership goals, seemed to require more time than the 
Council had, even though they spent more than eight hours considering the 
LRPC recommendation So 

3. Other decisions needed more data than the policymakers had at 
the time they had to make the decision. For example, membership direction 
decisions needed data about the interests and needs of AERA members and of 
others in research on education. Decisions about publications needed cost 
figures which were unavailable at the time. 

The LRPC Report was not considered final. Therefore some Council 
members were reluctant to take official actions on the recommendations in 
it. 

The decision-making structure was not very well integrated. 

The three-part influence process portrayed in Figure 1 was not as conducive 
as it might have been to the transmission of new 1 ideas , As one LRPC member 
observed, "I found the whole exercise fragmented. Communication was diffi- 
cult between the LRPC and the Council." 

VIII. SUMMARY 

This evaluation of the impact of the Colloquium on AERA policy- 
makingi suggests that a number of ideas emerged from the Colloquium which 
were relevant to the problems AREA decision-makers face. A number of these 
ideas reappeared in the LRPC Report and were subsequently considered by the 
AERA Council. In addition, the Colloquium seemed to raise more sharply 
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questions of identity and purpose for the AERA 0 Furthermore, certain ideas 
and questions raised by the Colloquium exist in present AERA activities, 
or are being considered by the Association. Finally, we speculated about 
why certain ideas seem to have been accepted by the Association while 
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others were noti 
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